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^ significant problem concerning the integration of dis 
play and switching functions is related to the fact that 
numerous informative data which have to be processed by man 
must be read from only a few display devices. A satisfactory 
ergonomic design of integrated display devices and keyboards 
is in many cases difficult, because not all functions which 
can be displayed and selected are simultaneously available. 

A technical solution which provides an integration of display 
and functional elements on the basis of the highest flexibility 
is obtained by using a cathode ray tube with a touch-sensitive 


screen. Aspects of sequential page selection for a virtual 
functional keyboard of an information management system are 
considered. The employment of an integrated data input/output, 
system is demonstrated for the cases of onboard and ground- 
based flight control. Ergonomic studies conducted to invest- 
igate the suitability of an employment of touch-sensitive 
screens are also discussed. 


17 . C« r (Selected ty Autrtor(«)) 


If. Ol«Trlbv<'«n 


Unclassified-Unlimited 


IT. f («l «M« <##i<lj 

30. Cltwlf. tM« r«o«l 

31. •( P«gif 

20 

« 

33. 

Unclassified 

Unclassified 





ERGONOMIC PROBLEMS REGARDING THE INTERACTIVE TOUCH INPUT 
VIA SCREENS IN ONBOARD AND GROUND-BASED FLIGHT CONTROL 

K. Holzhausen and K. Gaertner 

Institute of Anthropotechnology, Werthhoven (Bonn) 
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Summary . /I* 

Data input into a system via a screen is realized here by 
means of interactive touch input in communication', in which the 
image surface itself serves as a functional element. In addition 
to the standard display widely used today, the objective is also 
to effect an integration of the functional elements - specifical- 
ly here into display surfaces. A direct interaction of the pilot 
with the system of air traffic control by direct touch of the 
radar screen is recommended in the case of the ground-based flight 
control with the computer-aided radar map generated without the 
use of further functional elements. An integrated system for data 
input/output is discussed on the basis of the activity of hier- 
archially organized menus of virtual keyboards by the constant 

♦Numbers in the margin indicate pagination in the foreign text. 
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increment and variety of onboard systems for reasons of control- 
lability and of floor space required of display and functional 
elements for information management in the application of a 
touch-sensitive screen in the aircraft. The communication con- 
cerning touch-sensitve screens are explained, based on case 
studies of the ground and onboard-based flight control, and re- 
sulting ergonomic problems in the design of such systems are in- 
dicated. 

1 . Intro due t i on 


Modern aircraft technology and a growing density of traffic 
in aviation necessitate a constant further development of the on- 
board and ground-based system of flight control-. The designing 
engineer finds himself subjected to restrictions in the onboard- 
based flight control, which are set by the spatial limitation of 
the available cockpit space for the placement of displays in the 
visibility of the pilot and for the pattern of functional elements 
in the reaching space. Higher .demands on modern flight equipment 
lead, however, to a constantly growing value of information to be 
shown and of corresponding manual system inputs. 

Allowance is made for the growing variety of f display in- 
formation by the use of multipurpose display systems, which offer 
such information on only one instrument previously distributed on 
several mostly electromechanical displays. The employment of 
screens additionally allows the possibility of also placing plas- 
tic presentations on one and the same display at the disposal of 
the pilot in addition to numerous symbolic and numerical informa- 
tion. This process is commonly designated as display integra- 
tion. 


The success of the display integration allows the experi- 
ment to appear justified to integrate even functional elements, 
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in order to save space, on the one hand, and to gain operational 
advantages in connection with an integrated display, on the other 
hand, in order to relieve the individual working flight control 
system. The aspect of the functional element integration can, in 
the opinion of the authors, also he considered advantageous in 
the ground-hased flight control at the screen image working area 
of the pilot. 

The functional element integration was, first of all, main- 
ly referring to switches and keyboard keys. Examples of switch 
and key integration are electromechanical keyboards [l,2l and 
switch arrangements [3l , which are integrated in blocks and in 
which several functional representations are assigned to each 
key, or to each switch. A suitable connection of concepts for 
the display and functional element integration brings further ad- 
vantages. That is achieved by switching fields with a commutat- 
able function, whose legend can be digitally commuted. A further 
step of integration is done with the use of cathode ray tubes as 
display elements and ‘virtual keyboards shown there as functional 
elements with the simultaneous use of a touch-sensitive screen /2 
for the direct interaction of the operator with the system, 
since key fields can be computer-aided, arranged completely free 
and lettered here. Such a concept can be realized by employing 
a powerful computer system that is no problem in control towers 
and is also realizable today in practical operation within the 
aircraft. 

It is especially important to suitably design the man- 
machine-interface from the ergonomic point of view with the 
opening of such versatile possibilities for information manage- 
ment. The design should not only take the ergonomic view points 
of an integration of the display and functional elements into 
consideration, but it must also contain interfacing software for 
computer system, which supports the individual with strategies, 
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alternatives, and controlling and monitoring programs (4). 

A serious problem in integrating display and switching func- 
tions exists, for instance, in the fact that numerous data to he 
processed by the individual is to be read and executed on demand 
of a few display and key fields. A satisfactorily ergonomic 
design of integrated display and key fields becomes difficult in 
many cases, since not all displayable and switchable functions 
can be available at the same time. It can thus happen that func- 
tions are not exactly available when they are needed. 

2 . A Touch-Sensitive Screen As the Basis for an Integrated Data 
Input/ Out put System • 1 

A technical solution, which enables the integration of dis- 
play and functional elements on the basis of the highest flex- 
ibility, exists in the employment of a cathode ray tube with a 
touch-sensitive screen image surface. As a result, it is a /3 
question of a screen .which can be graphically programmed with 
computer support and whose size should conform to the respective 
application. A device for the decoding of screen positions is 
mounted on the screen surface, or around it, which signals each 
screen touch to the computer system as digital coordinates. A 
possible technical design for the decoding of touch inputs con- 
sists of a setting of respectively alternate matrix-shaped infra- 
red diodes designed with a photodetector, which are tested in the 
time multiplex as light barriers with a sufficiently compact de- 
sign. If these light rays are interrupted by any one object, 
e.g. the operator’s index finger, then this point is digitalized 
[5,6]. Touch inputs can be evaluated on such a screen surface 
as an actuation of supported virtual keys, or as simulation 
processes in plastic representations. 

The application of a touch-sensitive screen enables the con- 
centration of a multitude of necessary functional elements and 
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display devices in an integrated data input/output system. This 
universal device holds the advantage in the fact that the display 
instrument represents also simultaneously the functional element. 
The possible direct interaction here with the computer system ob- 
scures the difference between displays and functional elements. 

The application of virtual keyboards requires the organiza- 
tion of relevant system switching functions in functional switch 
designs, which are to be stored necessarily in the case of a 
greater variety on "pages" and "chapters" of an "electronic book" 
[71 . Each page of such a book represents a functional keyboard, 
which is connected with the respectively pertinent display in- 
formation. By "paging" in this book, the deBired page can be 
called up or cancelled, according to the desire- of the operator. 

In Figure 1, the initial structure of a page selection is basical- 
ly shown for an information management system. The Main Dis- 
patcher delineates the highest phase of the hierarchy, which 
enables the direct selection of the Subdispatcher of the next 
lower phase. From this point the direction leads further to the 
"chapters", which must be sequentially searched page by page /A 
for the opening of a "page". The return from a page to the Main 
Dispatcher should occur only as possible, as shown in Figure 1, 
with an interruption via the Subdispatcher. t 

3* Employment of an Integrated Data Input/Output System Is 
Demonstrated for the Case of Onboard-Based Flight Control 

To be able to monitor and to operate the numerous systems 
of a modern aircraft via a single data input/output system, very 
multifarious virtual key fields and graphic representations are 
used. Figures 2a and b each represent a virtual numerical and 
a functional key field. 
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Figure 2 : Application of an integrated data input/output, in- 
strument in the application of a touch-sensitive 
screen in the example of an aircraft management 
system: 
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a) Data input via a numerical keyboard 

b) Functional keyboard for the mode selection 

c) Computer-monitored tasking of a checklist 

Figure 2c shows the section of a map, which functions as a 
window. The map can be continuously shifted or stopped by func- 
V tional keys. A search for controlled information as well as the 
expansion of interesting image details is furthermore possible. 

As shown in the illustration, flight path manipulations are 
thereby directly visible for the pilot while he communicates 
with the system and give him a comprehensive impression of his 
flight situation. Figure 2d illustrates the applicable case of 
a checklist. The list is searched point by point .by the operator 
and checked off by touching the corresponding . virtual key. 

The illustrated series 3a - i is supposed to illustrate its 
performance with the simulated data input/output system at the 
Forschungsinstitut fuer Anthropotechnik (FAT) [7] . The operator 
begins with the Home keyboard (Main Dispatcher), as shown in /6 
Figure 3a. It always returns to the location of the respective 
functional keyboard in this Main Dispatcher. A series of systems 
can be called up via key pressure. Among others, there are 
(Figure 3a) the subsystems Engine and Fuel Management (ENG), con- 
trol of other contained Systems On Board (SYS), Frequency Option 
of the Navigational Receiver (NAV), Frequency Selection of the 
Transceiver of Radio telephony (COM), Checklist Processing (CHK) 
and Area Navigation System (ANS). 

The call-up and the utilization of one of these systems in 
the operation of a data input/output system with touch input is 
suppose to be illustrated by an example in the following. The 
Area Navigation System (ANS) of an aircraft is activated, in 
order to add to the programmed flight path, which consists of 
three defined path points, a new fourth path point. The system 
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simulation is based on Litton' s Area Navigation System LTN 72-R 
[8], which is used with an electromechanical keyboard in Luft- 
hansa aircraft. 

A small infra-red light emitting diode can be seen in the 
illustrations on the index finger of the operator. This is 
part of a Selspot System [9] , with whose aid the three-dimension- 
al movement of the index finger is decoded by a design of two 
optoelectronic cameras with rear-positioned digitalization elec- 
tronics. This design was selected in order to be able also 
to implement a dynamic survey of the trajectory travelled by the 
human finger next to the local coordinates on the screen [10] . 

After touching the ANS-key of the Main Dispatcher (Figure 
3a), the corresponding Subdispatcher appears (Figures 3b, 2b). 

By this method the ANS-system is selected. The diplay field 
can be seen in the upper portion of the illustration, on which 
the selected path point coordinates are later recorded. The six 
virtual keys located in the middle serve the Mode Selection and 
are used for the status display of the system in functional 
actions. The underlying functional keyboard permits the /7 
selection of ANS-relevant functions. At the bottom on the 
right, the Home-key is always located on every "p^ge Of this 
electronic book", which permits the operator at all times to re- 
turn from any page to the ANS-Subdi spate her and from there to 
the Main Dispatcher. The procedure of the path point input is 
demonstrated in employing the Map Mode instead of the numerical 
input of the local coordinates (List Mode). For this purpose 
the List/)VIap-key is, first of all, actuated (Figure 3b), in 
order to shift the system into the Map Mode. The key shows, 
subsequently, the inscription MAP in order to indicate the im- 
plemented shift. Subsequently, the path point key WP is act- 
uated (Figure 3c). The code WP appears at the top on the right 
above the display field. The ANS-Subdispatcher is automatically 
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Figure 3t Simulation case study of an onboard— "based integrated 
data input/output system in the application of a 
touch-sensitive screen 

Selection of the Area Navigation System and input of 
a fourth path point in the flight map with control 
of the path point coordinates in the display field /8 













replaced by a numerical keyboard to enable • the input of the 
number of the path point to be manipulated (Figure 3d), In this 
case, a 4 is input, which, subsequently, is communicated to the 
system via the input end key INS (Figure 3e), and an entry ap- 
pears on the path point key as well as across the display field. 
The flight map is simultaneously plotted with the preprogrammed 
flight plan (Figure 3f). In the next step, the pilot inputs the 
new path point WP 4 not yet defined in the flight plan by touch- 
ing the corresponding screen position (Figure 3g) . After that, 
the symbol WP 4 is entered by the system and the planned flight 
path extension is indicated by a dotted line from path point 3 
to path point 4. The entry of the new path point has been con- 
firmed by the system. By touching the Entry key (Figure 3h), 
the system receives this information as bindihgly communicated 
and indicates the coordinates of the path point to the pilot in 
digital form for control (Figure 3i). By one touch of the Home 
key, the display format returns to the ANS-Subdispatcher and by 
a further touch of this key, returns to the Main Dispatcher of 
the integrated data input/output system. /g 

4. Application of an Integrated Data Input/Output System 
Demonstrated for the Case of Ground-Based Flight Control 


In the case of ground-based flight control, -£he require- 
ment exists at the working area of the flight safety pilot at 
radar opertions to recall data concerning controlled flights from 
the system and to communicate its decisions to the system by in- 
puts. A summary of displays and functional elements is possible 
here as in the aircraft, while the entire surface of the screen 
is implemented as a touch-sensitive area of high resolution and 
thereby as a functional element. Additional corresponding vir- 
tual key fields are presented for functional selection and data 
input(ll]. That is possible - unlike in the aircraft cockpit - 
since screens are generally used with a diameter of about 50 
inches at air traffic control operations. /10 
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Figure 4:- Simulated radar map with a virtual functional keyboard 

Figure 4 shows a simulated radar map and, additionally, a 
virtual keyboard at the lower right edge of the screen. The sim- 
ulated map corresponds in simplified form to the radar map of one 
part of the upper air space controlled by EUROCONTROL in Maast- 
richt over Northern Europe [12, 13] • 

Outside of flight paths and warning points, air restricted 
territories, sector boundaries and other map data are plotted. 
Flight destinations are characterized by symbols with appending 
character fields, whereby an aircraft characteristic and command- 
ed air surface are contained in the simplified simulation. Such 
a radar map serves the pilots of EUROCONTROL for their work with 
the air traffic control system of MADAP, in the course of which 
the virtual keyboard was added in the simulation. 
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The virtual functional keyboard contributes here to the 
selection of a few operating methods, which are activated in the 
MADAP system over functional elements, or are not used by the 
pilot in this form. Such functions apply, in part, to the stan- 
dardization of the map scale (INC, CMP, ZOM) and displacement of 
map segments (SCR) and, in part, to isolated flight destinations, 
schedule and other information in question (LEV, EXP, RER, QDM, 
etc.) [llj. 

The illustrated series 5a - f is supposed to show, for ex- 
ample, a schedule change could be effected for a definite flight 
destination with a system of that kind by direct screen inter- 
action and use of a virtual functional keyboard. Since schedules 
are normally transmitted by the pilot at air traffic control 
before the start of a flight and are input to the computer system 
system there, they are recallable at any time by the pilot. Traf- 
fic situations, meteorological or other reasons require possibly 
the change of such plans during the flight. 

First of all, the pilot can be issued the schedule. For this 
purpose, he touches the screen position, on which the flight des- 
tination symbol is located at the time (Figure 5a) and, subse- 
quently, the virtual functional key EXP for the course extrapol- 
ation (Figure 5b). The schedule stored away in the air traffic 
control system is then represented as a dash-and-dot line /II 
on the map (Figure 5b). The directional line dims again as soon 
as the switch-off key is actuated, or a new operation is initiat- 
ed. Such a new operation is now the input of the changed 
schedule. After renewed designation of the flight destination 
symbol with the finger, the key RER is now actuated for the act- 
ivation of the reroute function, consequently, for the new input 
of the flight schedule. The system now waits for the input of 
the new schedule by designating warning points. In the figures 
5d - f, new directional corner points are keyboarded. Each new 
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flight schedule automatically begins in the current position of 
the flight destination symbol and is plotted by a dash-and-dot 
line along the new planned flight path. 

In the report, a film is shown for the demonstration of the 
touch input in conjunction with a data input/output system, which 
shows selected demonstrations of the interactive communication 
in data input/output systems in the aircraft cockpit and in the 
ground-based air traffic control. 

5 . Ergonomic Investigations and Outlook 

The first anthropotechnical experiments were conducted for 
proof of the basic suitability of touch-sensitive screens for 
the input of information input into systems. 

On one screen a destination range could be simulated from 
time to time lighting up at a non-foreseeable screen position as 
quickly as possible and with precision against given odds, i.e. 
could be reached with a cursor. A pivoting ball and a joystick 
as well as a radio direction finder and the touch input were 
used as functional elements. 

t 

The time requirements for the chosen task was practically 
the same for the radio direction finder and for the touch input 
but significantly greater, on the other hand, for the joystick 
and the pivoting ball. For that reason, it seems sensible 
to also consider the touch input for designated functions in 
addition to other functional elements [11} . /13 

The fact to be evaluated as an advantage of the touch input 
is that input operations are possible at any time without pre- 
viously accepting functional elements at hand. This fact always 
plays a part when the operator still has to safeguard a number 
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of other functions in addition to the inputs. 


It was attempted in further experiments to establish suit- 
able key sizes for the actuation of virtual keyboards. For this 
purpose, a numerical keyboard was presented to researchers on 
the screen, above which was located a display and an input field. 
The keyboard is similar to the keyboard represented in Figure 2a. 
In the display field a four-digit combination, beginning with a 
0, appeared, which was, subsequently, to be input. The digits 
keyboarded for the operator were presented in the input field. 

A correction key was located next to the numerical keyboard, 
by whose actuation the inputted digit could ultimately be deleted. 
The key was designed under the correction key for the continuous 
circuit arrangement of the experiment, by which the next combin- 
ation of digits was recalled. Quadratic virtual keys were used 
with a lateral length of 16, 22 and 40 mm as the key size. The 
spaces between the keys totalled 2, 3 and 5 mm. 

From time to time, 25 digital combinations per key size 
could be input with four experimental repetitions. There was 
concern with these experiments in implementing the function 
quickly and safely. Before the researcher began v^lth the digit- 
al keyboarding, he had to accurately memorize the impending com- 
bination of digits. The timing did not begin unti after the 
touching of the first digit and ended after entry of all four 
digits. In case of inputting wrong digits, or of touching the 
key extremities between two keys, a greatly increased time re- 
quirement was generally recorded for input correction. 

As was to be expected after the preceded designation ex- 
periments, an enlargement of the keys from 16 to 22 mm led to a 
diminution of the necessary input times. Interestingly, the key 
enlargement produced no further time savings at a 40-mm corner 
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length, hut rather an Increasing time requirement sooner. That 
was apparently based on the fact that the researchers tended /14 
to fully utilize the entire key field while using larger keys. 
Faster inputs were possible by the limited requirements of ac- 
curacy, which the researchers subjectively felt to be relaxing. 
This advantage was, of course, reduced by the increased times 
the hand moved from key to key. The employment of such large 
keys is only to be recommended in that case when the aspect of 
faster and more frequent inputs is not a factor. 

The acknowledgment of key touchings for the researcher was 
investigated as a further problem. The operator already received 
visual acknowledgments via the input by delineating a cursor on 
the touched screen position in the simulated experiments. This 
kind of acknowledgment was already welcomed by all researchers 
in the preceded experiments and also used here. It was not valid 
as an experimental parameter, because a cursor is unrelinquish- 
able in every kind of screen positioning, especially, in the ap- 
plication of indirect screen functional elements like the pivoting 
ball or the joystick. Virtual keys have again the basic charact- 
eristic of leaving out the accustomed tactile feeling as a re- 
stored signal for the methodical actuation of a key when mani- 
pulating the keys. That is why ergonomic investigations are 
necessary in the use of that kind of virtual key fields via 
alternative solutions for a restored indication supporting the 
functional method. 

For this purpose, practically only auditive acknowledgments 
are suitable. The above-described digital input experiments were 
repeated for this reason for a series of different auditive ack- 
nowledgments: after the experimental series without acoustic 
input abandonment, a further series was implemented, in which 
each methodical key actuation, i.e. each key actuation meets 
without jointly touching an adjacent key with a 1000-Hz touch 
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tone. In case of error, this acknowledgment was absent. A 
further experimental series containted no acoustic acknowledg- 
ment in the case of a normal actuation, but a tone combination /15 
perceived as dissonant in case of error (false tone). 

In the light of data material gained by only three research- 
ers, an effect of these different auditive acknowledgments can 
hardly be recognized in actuation times in the case of largely 
correct inputs. Somewhat unexpected, though, was the result 
that the touch tone was not fully rated affirmatively in the 
multitude of inputs. Eather, a tendency existed to perceive the 
touch tone to be unpleasant or even disturbing. From this, it 
is to be considered whether not the visual acknowledgment will 
suffice by the cursor dagger. A further effect resulted in an 
occasionally observed insignificant increment of the input times, 
when the touch tone was synchronized. The researchers admitted 
that the tone would have forced a kind of rhythmic coordination 
during the operation, which subjectively would be contradictory 
to the experimental purpose of the optimal time input and would 
lead sooner to a rhythmical operational execution. 

The variable functional characteristics in the case of er- 
ror conditions are of interest. Without a false tone, the tend- 
ency exists, first of all, to completely input the digital com- 
binations without taking notice of wrong inputs. Only afterwards 
does the visual control of the input and the comparison with the 
display occur, after which the correction process begins. In the 
case of the false tone open circuit arrangement, the input pro- 
cess is generally, however, immediately interrupted by the op- 
erator. Because basically none of the two touched key fields is 
rated as input in an error operation, the operator could now im- 
mediately repeat the next display digit and the input will cor- 
rectly lead to an end. As a result of the false tone open cir- 
cuit arrangement, a somewhat faster execution of error-affected 
inputs was then to be recorded, which subjecively gave the op- 
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erator a greater measure of security. 

Future experiments should help to safeguard discovered tend- 
encies and to prepare investigations concerning practicable menu 
structures . 
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